Tricky Back Propagation

Posted on | In

Some operations are indifferentiable, which causes difficulties for back propagation.

  1. sample from distribution: reparameterization trick
    Auto-Encoding Variational Bayes

  2. argmax: soft argmax
    Gradient Descent Optimization of Smoothed Information Retrieval Metrics

  3. crop: two-dimension boxcar function
    Look Closer to See Better: Recurrent Attention Convolutional Neural Network for Fine-grained Image Recognition

  4. delta function: lookup table+blur kernel
    Reblur2Deblur: Deblurring Videos via Self-Supervised Learning